Assessing cryptographic entropy

ABSTRACT

Systems, methods, software, and combinations thereof for evaluating entropy in a cryptography system are described. In some aspects, sample values are produced by an entropy source system. A typicality can be determined for each of the sample values. A grading is determined for preselected distributions based on the typicalities of the sample values. A subset of the preselected distributions are selected based on the gradings. An entropy of the entropy source system is calculated based on the subset of the plurality of distributions.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of and claims priority to U.S. patentapplication Ser. No. 13/307,078, filed on Nov. 30, 2011, the entirecontents of which are hereby incorporated by reference.

BACKGROUND

This specification relates to assessing entropy in a cryptographysystem. Cryptography systems enable secure communication over publicchannels. For example, in public-key cryptography systems, a messagesender encrypts a plaintext message using an encryption algorithm, andtransmits the encrypted message to a recipient. The recipient can use asecret key value to recover the plaintext message from the encryptedmessage. In some cryptography systems, the entropy of the secret keyvalue makes the cryptography system robust against certain types ofattacks.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram of an example cryptography system.

FIG. 2 is an example method for evaluating entropy in a cryptographysystem.

DETAILED DESCRIPTION

A cryptography system can generate and utilize cryptographic secrets(e.g., secret key values, passwords, etc.). In some instances,cryptographic secrets can include entropy from an entropy source. Forexample, a pseudorandom number generator that generates cryptographicsecrets can be seeded by a value that is random, from an adversary'sperspective, to protect against certain types of attacks by theadversary. Entropy can be used in a combination of these and other waysto decrease the effectiveness or efficiency of sophisticated adversarialattacks.

This disclosure describes several forms of entropy appropriate for usein cryptography systems. This disclosure also describes statisticalinference techniques appropriate for assessing entropy in cryptographysystems. For example, cryptographic keys include entropy to avoid beingguessed by adversaries. The entropy gathered for the keys can beassessed, for example, so that estimation for risks of attacks can bereliably performed. The present disclosure describes example entropysources that can be used for some cryptography systems. The presentdisclosure also describes some example assumptions about theprobabilities characterizing the entropy sources. In some aspects of thepresent disclosure, a conservative estimate of the amount of entropy isproduced by using statistical inferences.

Multiple types of entropy assessments can be made. In some instances,e.g., to generate a conservative assessment, the minimum of theassessments can be taken as the total or actual assessment. The entropycan be assessed based on sample values obtained directly from an entropysource. In some instances, entropy measurements based on values obtainedfrom the entropy source provide a more accurate or more usefulassessment than other types of measurements, such as, for example,entropy measurements based on the output of a cryptographic operation.Some cryptographic operations, such as hash functions and block ciphers,can be generally designed to produce outputs which appear uniformlydistributed even if the inputs are non-uniform. An inference method thatdoes not specifically account for the cryptographic operations canpotentially, in some instances, yield an assessed entropy exceeding theactual entropy. In such instances, the task of entropy assessment canpotentially be obscured or even undermined. In some implementations,entropy assessments that account for cryptographic operations may beused.

FIG. 1 is a schematic diagram of an example cryptography system 100. Theexample cryptography system 100 includes an entropy source system 102, acryptographic secret generator module 106, an entropy assessor module108, and a cryptographic communication module 110 a. In some instances,as in the example shown in FIG. 1, the cryptography system 100 includesadditional cryptographic communication modules (e.g., the cryptographiccommunication module 110 b, and possibly more) and an adversary 112. Thecryptography system 100 can include additional or different features andcomponents, and the cryptography system 100 may be configured as shownand described with respect to FIG. 1 or in a different manner.

In one example aspect of operation, the entropy source system 102provides output values to the cryptographic secret generator module 106.The output values provided by the entropy source system 102 can serve asa source of entropy for the cryptographic secret generator module 106.For example, the cryptographic secret generator module 106 can be apseudorandom number generator module that is seeded by the output valuesfrom the entropy source system 102. The cryptographic secret generatormodule 106 produces a cryptographic secret (e.g., a private key) andprovides the cryptographic secret to the cryptographic communicationmodule 110 a. The cryptographic communication module 110 a communicateswith other entities in the cryptography system 100 based on thecryptographic secret. For example, the cryptographic communicationmodule 110 a may produce encrypted messages, digital signatures, digitalcertificates, cryptographic keys (e.g., public and private keys,ephemeral and long-term keys, etc.), or a combination of these and othertypes of cryptographic data based on the secret key provided by thecryptographic secret generator module 106. The secret value provided bythe cryptographic secret generator module 106 can be used as, or can beused to produce, a cryptographic key for cryptographic communications.

In some aspects of operation, the entropy assessor module 108 assessesthe entropy of output values produced by the entropy source system 102.For example, the entropy assessor module 108 can obtain sample outputvalues from the entropy source system and assess the entropy of theentropy source system 102 from the perspective of the example adversary112, or from another perspective. In some cases, the entropy assessormodule 108 uses one or more of the techniques described below. Theentropy assessor module 108 may evaluate the entropy, for example, inconnection with manufacturing the entropy source system 102, thecryptographic secret generator module 106 or any combination of theseand other components of the cryptography system 100. The entropyassessor module 108 may evaluate the entropy, for example, in connectionwith deploying or testing aspects or components of the cryptographysystem 100, during use of a cryptography system 100, or in otherinstances. In some implementations, the entropy assessor module 108evaluates the entropy of the entropy source system 102 when the entropysource system 102 is instructed to provide output values for thecryptographic secret generator module 106.

The components of the example cryptography system 100 shown in FIG. 1can be implemented in any suitable combination of hardware, software,firmware, or combinations thereof. In some instances, the entropy sourcesystem 102, the cryptographic secret generator module 106, the entropyassessor module 108, and the cryptographic communication module 110 acan be implemented as software modules that are executed by one or moregeneral purpose processors. In some instances, one or more of theentropy source system 102, the cryptographic secret generator module106, the entropy assessor module 108, or the cryptographic communicationmodule 110 a can be implemented as one or more hardware components. Thehardware components may include specialized processors or pre-programmedlogic, general purpose processors executing software, or other types ofdata processing apparatus.

The cryptographic secret generator module 106 produces an output basedon values provided from the entropy source system 102. For example, thecryptographic secret generator module 106 can include a pseudorandomnumber generator or another type of system or process. A pseudorandomnumber generator can be implemented in hardware, software, firmware, orany combination thereof. A pseudorandom number generator can operatedeterministically and provide an output that appears random from theadversary's perspective. Accordingly, the output of a pseudorandomnumber generator can be referred to as a random output value althoughthe pseudorandom number generator itself operates deterministically. Insome implementations, the output value obtained from a pseudorandomnumber generator depends on the state of the pseudorandom numbergenerator. For example, the pseudorandom number generator may definemultiple different states that each correspond to a different outputvalues. The state can be updated, for example, by progressing to asubsequent state, each time the pseudorandom number generator providesan output value. The state can be refreshed (or reseeded), for example,to add entropy to the state periodically or upon specified conditions.

The cryptographic communication module 110 a can include any suitablehardware, software, firmware, or combinations thereof, operable toexecute cryptographic operations. In some instances, the cryptographiccommunication module 110 a is configured to perform data encryption. Forexample, the cryptographic communication module 110 a may be configuredto encrypt messages or other types of data based on a secret keyprovided by the cryptographic secret generator module 106. In someinstances, the cryptographic communication module 110 a is configured toprovide data authentication. For example, the cryptographiccommunication module 110 a may be configured to generate a digitalsignature or authentication tag based on a secret key provided by thecryptographic secret generator module 106. In some instances, thecryptographic communication module 110 a is configured to generatedigital certificates or other types of cryptographic objects. Forexample, the cryptographic communication module 110 a may be configuredas a certificate authority to issue digital certificates based on asecret key provided by the cryptographic secret generator module 106.The cryptographic communication module 110 a may be configured toperform additional or different types of operations.

In some implementations, the cryptographic communication modules 110 aand 110 b can communicate with each other over an open channel. Forexample, the cryptographic communication module 110 a may sendcryptographic data (e.g., encrypted messages, signed messages,cryptographic certificates, public keys, key-agreement data, etc.) tothe cryptographic communication module 110 b over a communicationchannel that is observable, partially or wholly, by the adversary 112.In some instances, the cryptographic communication modules 110 a and 110b can communicate over one or more data communication networks, overwireless or wired communication links, or other types of communicationchannels. A communication network may include, for example, a cellularnetwork, a telecommunications network, an enterprise network, anapplication-specific public network, a Local Area Network (LAN), a WideArea Network (WAN), a private network, a public network (such as theInternet), a WiFi network, a network that includes a satellite link, oranother type of data communication network. Communication links mayinclude wired or contact-based communication links, short-range wirelesscommunication links (e.g., BLUETOOTH®, optical, NFC, etc.), or anysuitable combination of these and other types of links.

The entropy source system 102 can include one or more entropy sources.For example, as illustrated in FIG. 1, the entropy source system 102includes two entropy sources 104 a and 104 b (“entropy sources 104a-b”). In some instances, entropy sources can be modified or added tothe entropy source system 102 as needed, for example, to achievesufficient entropy. The entropy sources 104 a-b can provide informationthat has entropy. For example, the entropy sources 104 a-b can include aring oscillator, a noisy diode, mouse movements, variances in disk readtime, system process usages, or other quantifiable but unpredictablephenomena or behaviors. In some implementations, the entropy sources 104a-b generate a collection of samples (e.g., logged signal, measurementhistory, saved variation) that are accumulated into an entropy pool byusing a deterministic process. For example, the entropy pool can be aconcatenation of all the samples values. Compression may be applied tothe concatenation on account of memory restrictions. The compressionprocess may be a group addition, a cryptographic hash function, a randomextraction, or any other appropriate compression method.

The cryptographic secret generator module 106 may use the entropy poolof the entropy sources 104 a-b to generate the cryptographic secret(i.e., keys). For example, a seed can be extracted from the entropy poolfor the secret generation. The cryptographic secret generator module 106can be operated as a well-seeded and well-designed deterministicpseudorandom number generator to generate random numbers as the keys.The initial seed can provide the cryptographic entropy to the numbergenerated. In some instances, the random numbers generated by thecryptographic secret generator module 106 can appear asindistinguishable from uniformly distributed random numbers. Thecryptographic secret generator module 106 may employ backtrackingresistance and other techniques. For example, output values can begenerated by a pseudorandom number generator in a manner that avoids theoutput values feasibly being used to recover the internal state of thepseudorandom number generator, and avoids the output values feasiblybeing used together with the internal state of the pseudorandom numbergenerator to determine past internal states. In some implementations,the backtracking resistance of the cryptographic secret generator module106 can provide forward secrecy of key agreement schemes.

The processing of the samples from the entropy sources 104 a-b to thecryptographic secret generator module 106, can be deterministic andwithout additional entropy. The deterministic algorithms in acryptography system may not be kept sufficiently secret, or it may beunrealistic in some instances to assess the entropy of an algorithm. Forexample, if the adversary does not know the algorithm, it may beunrealistic in some instances to measure or quantify the informationthat is unknown to the adversary as a result of the adversary notknowing the algorithm. The entropy sources 104 a-b may behave accordingto a probability distribution that can be used to assess the entropysources 104 a-b. This probability distribution may be less than exactlydefined, but can be assumed to at least belong to some known set ofprobability distributions. Under such assumption, statistical inferencescan be applied to assess the cryptographic entropy provided by theentropy sources 104 a-b, and/or other entropy sources in general.

In some implementations, the entropy sources 104 a-b include or accessprocesses of an operating system (e.g. the operating system of acomputing device). In some implementations, for software to have anentropy source, one common practice is to examine the set of processesrunning on a computer on which the software is installed. In someexample operating systems where multiple processes are sharing processortime, the list of processes with amount of processor time each has used,may have some entropy. For example, some processes may need to write toa hard disk. When writing to a hard disk, the disk-seek-time is known tovary depending on where data is located on the hard disk and otherfactors. An advantage of such entropy sources is that special hardwareor user actions are not required for the entropy source.

In some implementations, the entropy sources 104 a-b include or accessenvironmental conditions. For example, some systems have inputs whichcan be used as an entropy source, such as a microphone for monitoringthe sound in the local environment. The audio recording may be acombination of noises and ambient activities that cannot be predicted.An advantage of such an entropy source is that the entropy source maynot require user actions or special or additional hardware (e.g.,because microphones, video cameras, and other computer peripheralhardware are commonly available). A possible disadvantage is anyadversary close enough may also have partial access to the entropysource. For example, the adversary 112 may place a microphone in thesame environment to record the sound of the same ambient activities.

In some implementations, the entropy sources 104 a-b include or accessuser inputs. In some systems, users often supply inputs, such as, forexample, mouse movements, keyboard strokes, touchpad gestures, etc.These inputs may be used as the entropy sources 104 a-b. The inputs usedfor entropy may be gathered incidentally through normal use or through aprocedure where the user is requested to enter inputs with theinstruction to produce something random. In addition to treating userinputs as an entropy source from which entropy can be extracted toderive a secret cryptographic key, a system can rely on the user todirectly provide a secret value, for example, in form of a user-selectedpassword. The user-selected passwords may require entropy. Thus, theentropy of user-selected passwords can be assessed. In someimplementations, system-generated passwords can also be used as thesecret value. The system-generated passwords can apply a deterministicfunction to the output of random number generator. The deterministicfunction renders the random value in a more user-friendly format, suchas alphanumeric. The result is a password that may need entropy, and thesource of entropy may be some other entropy source. Thus, the entropy ofthe password can be assessed.

In some implementations, the entropy sources 104 a-b may include oraccess coin flipping. In some examples, the coin flipping entropy sourcemay be realized as follows. A coin is thrown by a person into the air,with some rotation about an axis passing nearly through a diameter ofthe coin. The coin is either allowed to land on some surface or to becaught by hand. A result is either heads or tails, determined by whichside of the flipped coin is facing up. Coin flips are often modeled suchthat each result is independent of all previous results. Furthermore,for a typical coin, it is often modeled that heads and tails are equallylikely. A sequence of coin flips can be converted to a bit string byconverting each result of head to a 1 and each tail to 0. In such asimple model, the resulting bit string is uniformly distributed amongall bit strings of the given length.

In some implementations, more skeptical and/or sophisticated models ofthe coin flipping can be formulated as follows. Firstly, it may be notedthat a dishonest coin flipper could potentially cheat in certain ways.For example, the cheater may not rotate the coin on the correct axis,but rather an axis at a 45 degree to the plane of the coin. This maycause the coin to appear to rotate, but maintain one side closest to aparticular direction in space. For another example, a cheater, withjuggling skills, may be able to toss the coin with a given speed androtation (of proper type) such that either the coin can be caught withan intended side up, or perhaps land on a surface with higherprobability of landing on an intended side. If one considers thatcheating is possible, then consider that an honest coin flipper mayinadvertently introduce bias in the coin flips. Indeed, in acryptographic application relying only on coin flips for entropy, a usermay need to flip a coin at least 128 times. As the user then becomestired of repeated flips, the user may start to become repetitive andperhaps suffer from such bias. To account for this, one could formulatea more pessimistic probability model for the coin flippers, and then dosome statistical analysis comparing the pessimistic model with theactual sample of coin flips.

In some implementations, the entropy sources 104 a-b may include oraccess one or more dice. Dice, or cubes with numbers on the sides, canbe used in games of chance. Provided that adequate procedures are usedin the rolling, the number that ends up at the top of the die, when itsmotion has ceased, is believed to at least be independent of previousevents. On the one hand, the roll of a die seems to be governed mainlyby deterministic laws of mechanics once it is released, so it may seemthat all the randomness is supplied by the hand that rolled the die. Onthe other hand, it may be apparent that the rollers of dice cannotcontrol the results of the dice rolls, and in particular the source ofthe randomness is the actual rolling process. This discrepancy can beexplained as follows.

Each collision of the die with the ground causes it to bounce. Becausethe die is tumbling as it bounces, some of the rotational energy of thedie may be converted into translational energy of the die, and viceversa. This conversion depends very much on the orientation of the dieas it impacts the surface upon which it rolls. With each bounce, theresulting translational energy affects the amount of time before thenext bounce. The amount of time between bounces affects the amount ofrotation of the die, and therefore its orientation. This may mean that asmall difference in orientations at one bounce results in a largedifference in orientations at the next bounce.

The behavior of a rolling die may be similar to the butterfly effect,because each bounce magnifies the effect of the orientation androtation, and the outcome of the die roll, as determined by the finalorientation of the die depends on the fine details in the initialorientation and motion of die. Such a process is known as chaotic.Although technically deterministic, physical chaotic processes aredifficult to predict, partly because it is impractical to obtain thenecessary precision on the initial conditions to determine the finalcondition. Therefore, rolling dice can be a practical way to seed arandom number generator that can be used to generate organizationallevel secret keys. Rolling dice may be impractical for some types ofuser-level secret keys, for example, where rolling dice is infeasiblefor runtime sources of entropy.

In some implementations, the entropy sources 104 a-b may include oraccess a ring oscillator, which has been a common source for entropy.Ring oscillators can be implemented as odd cycles of delayed not-gates.Whereas even cycles of delayed not gates can be used for memory storage,ring oscillators tend to oscillate between 0 and 1 (low and highvoltage) at a rate proportional to the number of gates in theoscillator. Since the average oscillation rate can be calculated fromthe number of gates and general environmental factors, such astemperature, the variations in the oscillation may be regarded as theentropy source. Ring oscillators are not available in some generalpurpose computer systems, but they can be included in other types ofhardware (e.g., custom systems, field programmable gate arrays (FPGA),etc).

In some implementations, the entropy sources 104 a-b may include oraccess a radioactive decay. Some smoke detectors use the radioactiveelement americium which emits alpha particles. The same method may beused as a cryptographic entropy source, such as for the generation oforganization-level secret keys.

In some implementations, the entropy sources 104 a-b may include ahypothetical muon measurer. A muon measurer can provide a 32-bit measureof the speed of each muon passing through the device. In some examples,one muon passes through the detector per minute on average. Because ofthe underlying physics of muons, this entropy source may be viewed asproviding a robust entropy source, whose rate of entropy cannot beaffected by an adversary.

In some implementations, the entropy sources 104 a-b may include oraccess a quantum particle measurement. The theory of quantum mechanicsimplies that quantum particles, such as photons or electrons, can existin a superposition of states under which measurement causes a wavefunction collapse. The theory states that wave function collapse is afully random process independent of all other events in the universe.Under this theory, an entropy source derived from such wave functioncollapse would be absolutely unpredictable, which is highly useful forcryptography.

The entropy sources 104 a-b can be assessed for reliable risk analysisso that the cryptographic secret generator module 106 can produce asecret key useful against adversaries. As an example, unclearresponsibility for properly seeding of pseudorandom number generatorscan result in problems. Suppose a manufacturer of cryptographic softwareimplements a pseudorandom number generator but does not provide a sourceof entropy. If the manufacturer sets the seed to a default value, anduser of the software may mistakenly generate “random” values with thedefault seed, unwittingly believing that random number generatorincludes a source of entropy, then the outputs of the pseudorandomnumber generator may be considered to have zero entropy (e.g., to theadversary who knows the default seed). An assessment of entropy may helpto avoid such vulnerabilities.

Formally assessing entropy can, in some instances, prevent or reduce thechances of certain types of failures and vulnerabilities in acryptography system. For example, if the internal state of adeterministic pseudorandom number generator is somehow revealed to anadversary, then all its future outputs can be determined by theadversary, unless the pseudorandom number generator is refreshed withnew entropy. The property obtained by frequent refreshing is sometimesreferred to as prediction resistance (where refreshing is sometimesreferred to as reseeding), or forward security. The entropy needed forforward security generally can be obtained during deployment in thefield. In many cases, entropy in the field may be regarded as scarce.For this reason, entropy assessment is appropriate.

Assessment on the entropy sources 104 a-b can be performed by theentropy assessor module 108 before deployment, during deployment, or atany appropriate combination of these and other instances. For example,one or more sample values from the entropy sources 104 a-b can be usedto infer something about their distribution. In some cases, the samplevalues are discarded, and the inferences about the sources are used toassess their ability to generate entropy in the future. This approachprovides prospective assessment, performed before deployment. In anothercase, the sample values can be used for some cryptographic application,such as forming some of the input to the cryptographic secret generatormodule 106, which derives a secret key. For example, there may beenvironments or contexts where entropy is believed to be so scarce thatit is not affordable to discard the sample values. This approachprovides retrospective assessment, performed during deployment. In somesituations, retrospective assessment can leak information to anadversary. As such, contingent entropy may need to be assessed, asappropriate.

In some situations, the cryptographic secret generator module 106 maygenerate keys for multiple uses such that one observation of such usageprovides enough information to determine, using unlimited computation,the key which could then be used to compromise the remaining usage ofthe key. For example, in many forms of public-key cryptography, a publickey determines uniquely its corresponding private key. As anotherexample, suppose a typical stream cipher (which attempts to generate aone-time pad from a finite key), such as the Advanced EncryptionStandard used in counter mode (AES-CTR), is used once, but on a message,part of which is known to the adversary and part of which is unknown. Ifthe known part of the message has sufficient length then, the adversary,given unlimited computation, could determine the stream cipher key, andthen the whole message. By contrast, some cryptographic protocols offerinformation-theoretic security. These protocols attempt to resist anadversary with unlimited computational power, by impractically requiringa very large cryptographic key, which in many cases can be nearlyuniform.

Keys whose continued security relies on computational assumptions mayhave the property of confirmability. An adversary who has the candidatekey can confirm the key's correctness by observing the actual use ofkey. This means that what one considers as the entropy of key needs toaccount for an adversary who can exhaustively search for keys. Sometypes of computational-security keys, such as public keys, are such thatpurely computational attacks are strictly faster than exhaustive searchof all possible values of the keys. For example, discrete logarithmkeys, such as those used in Diffie-Hellman key agreement or El Gamalsignatures, keys may be positive integers less than some prime q.Algorithms, such as Pollard's rho algorithm, can compute the private keyabout √{square root over (q)} steps. Schnorr gives strong evidence thatif the private key is chosen from a random set of size √{square rootover (q)}, which allows for exhaustive search of √{square root over (q)}steps, no significant improvement of generic algorithms, such as Pollardrho, is any faster than about √{square root over (q)} steps. As such,some discrete logarithm private keys may require about half as muchentropy as the bit length.

Other types of computational-security keys, such as symmetric encryptionkeys, are such that currently-known computational attacks have costsimilar to exhaustive search. For example, consider the block cipherAdvance Encryption Standard (AES), with a key size of 128 bits. Attackson the use of block cipher exhaustively search each possible key.Accordingly, AES-128 is generally considered to provide 128 bits ofsecurity. But providing 128 bits of security seems to require the key beperfectly uniform, meaning it has 128, or almost 128, bits of entropy.Such considerations provide incentive to generate uniform keys. Creatinga nearly uniform distribution from biased entropy obtained from actualsources in environments where entropy is scarce may be too expensive insome contexts. Nevertheless, suppose that AES-128 was used with keyshaving only 100 bits of entropy. Presumably, then only 100 bits ofsecurity would be provided. Some chance exists that such keys could beweak. But this would seem unlikely if the keys were selectedpseudorandomly, such as by the output of a hash. If 100 bits of securityprovide adequate protection, then the burden of producing a uniform keyis lifted, and one can concentrate on providing adequate entropy.

The entropy assessor module 108 can be used by third parties or othertypes of entities. In some situations, when a first party supplies acryptographic product to a second party, the second party values a thirdparty evaluation of the cryptographic product. Third party evaluationsof entropy can have some difficulties. For example, proper entropyassessment requires direct access to the sources, which normally, may benot available in the cryptographic product. The first party has anincentive to supply the output of a deterministic pseudorandom numbergenerator as the claimed source. To a third-party evaluator, the effectof this would be the source appears to adhere to a uniform distribution.

The entropy assessor module 108 can be used within an organization oranother type of secure infrastructure. In some situations, anorganization may provide its members secret keys for encryptionpurposes, but retain a backup copy of the secret keys. The organizationmay use a deterministic pseudorandom number generator to generate secretkeys for members. The organization may need to be sure about thesecurity of the secret keys, and would likely invest considerableresources into using sufficient entropy for the seed. Some cryptographicapplications, such as personal privacy and non-repudiation, require thata user's secret key be truly secret to the user. In this case, someentropy for the user's secret key can be generated on the user's localsystem.

In some situations, user-defined passwords are values that a user needsto recall and enter into a device, usually to authenticate the user forreasons such as to obtain permission to access certain privilegedinformation. Such passwords are typically too short to contain enoughentropy to be used as cryptographic secret key in the sense of beingable to render exhaustive search infeasible. This shortness is partiallybased on the belief that users cannot remember high-entropy passwords.Because of their low entropy, any data value which would allow off-lineconfirmation of the password guesses, such as the hash of a password,may not be made public. Otherwise a password-dictionary could beapplied. Password-authenticated key agreement schemes, such as SimplePassword Exponential Key Exchange (SPEKE), are designed to avoid suchoff-line attacks. The restriction on the exposing user-defined passwordsto off-line guessing attacks can apply to both user-selected andsystem-generated passwords.

In some example aspects of operation, the entropy assessor module 108can assess whether a cryptographic secret is guessable by the exampleadversary 112. The entropy assessor module 108 may observe sample outputvalues from the entropy sources 104 a-b. The entropy assessor module 108may rely on a hypothesized probability model for the possibledistributions of the entropy sources 104 a-b. The probability model canbe hypothesized based on theoretical understanding of the entropysources 104 a-b, observations of sample output value from the entropysources 104 a-b, or any suitable combination of these and otherinformation. The entropy assessor module 108 may infer, from theobserved sample output values produced by the entropy sources 104 a-b, aset of distributions for the entropy sources 104 a-b within thehypothesized probability model. The set of distributions can be inferredby considering a grading between sample output values and thedistributions. The grading can be based on likelihood, typicality, or acombination of these and others. The set of distributions can beinferred based on maximizing or thresholding the grading from theobserved sample values.

In such example aspects of operation, the gradings may be induced from asample statistic. The sample statistic may include the hypothesizedprobability model. For example, the sample statistic can bemodel-neutral or model-invariant. The probability model may be a Markovmodel. The gradings may be induced from the sample statistic. In someexamples, the sample statistic can be the Markov frequency statistic,and the induced probabilities can be computed using the Goulden-Jacksonformula. The typicality grading may be determined in terms of a sum ofthe probabilities of the samples that are less than or equal to theprobability of the sample being graded. The typicality grading may bedetermined in terms of a probability-weighted sum of all over the samplevalues of an odd function comparing the probability of each sampleoutput value with the probability of the sample being graded. The oddfunction comparing the sample probabilities can be linear.

In some example aspects of operation, the entropy assessor module 108may quantify the entropy or unguessability of the entropy sources 104a-b. For example, by the entropy assessor module 108 may quantify theentropy of the entropy sources 104 a-b by characterizing the workload ofthe adversary 112; by determining the possible side channel informationleaked to the adversary 112; by considering what function of the samplevalues will be used in derivation of a cryptographic key; bydetermining, for each distribution, a value of the conditional appliedworking entropy of the distribution (e.g., taking the logarithm of theadversary's maximal probability of guessing the cryptographic key); bytaking the minimal value of the entropy over the inferred set ofdistributions; or by any suitable combination of such techniques.

In some example aspects of operation, the entropy assessor module 108may take steps to ensure that the total entropy of the values used bythe cryptographic secret generator module 106 to derive thecryptographic secret is sufficient for a specified level of security.For example, the entropy assessor module 108 may determine from thespecified security level, an adequate level of entropy at the givenworkload characterized for the adversary 112. In some instances, if theassessed entropy is inadequate, then the entropy assessor module 108 mayobtain more samples from the entropy sources 104 a-b or obtain moreentropy sources, and reassess the entropy until the amount of entropy isadequate for the desired security level. In so doing, the entropyassessor module 108 may account for the possibility that the processused to make decisions may leak some portion of this information to theadversary 112. In some instances, a cryptographic key can be derivedfrom samples obtained from the sources.

In such example aspects of operation, assessment of the entropy may beperformed prospectively. For example, the sources may be assessed usingobserved sample values from the source before any key generation is tobe done, such as time of development. Assessment may be performedretrospectively in the sense that the sources may be assessed using, inpart, sample values that are intended to be used for key generation. Thecalculated entropy may be eventuated entropy that accounts for thepossibility that the samples obtained from the source may have a lowerprobability, and therefore more entropy, than the most probable outputsof the source.

In some example aspects of operation, the minimization of the entropywith respect to the inferred set of distributions, and the possiblemaximization of the grading to obtain the inferred set of distributions,may use a generic optimization technique. The generic optimizationtechnique can be, for example, a gradient method, Newton's method,conjugate direction methods, quasi-Newton methods, neural networks,non-differentiating methods, such as Nelder-Mead simplex or simulatedannealing, penalty methods, Karush-Kuhn-Tucker conditions, convexmethods. The solution to one of the optimization problems involved inthe entropy assessment, such as minimization of the entropy over theinferred set of distribution or the determination of the inferred set ofdistributions as a maximally graded set, may be obtained through analgorithm specific to the hypothesized probability model. For example,if the hypothesized probability model is the hidden Markov model, theBaum-Welch algorithm may be used to solve the optimization probleminvolved in entropy assessment.

FIG. 2 is a flow chart showing an example process 200 for calculatingentropy in a cryptography system. The example process 200 can beexecuted by the entropy assessor module 108, for example, to assess theentropy of the entropy sources 104 a-b shown in FIG. 1. Each of theentropy sources 104 a-b can be assessed individually. The entropy ofindependent sources can be combined (e.g., added together or combined inanother manner). The entropy source system 102 can be assessed as anentropy pool. In some instances, some or all of the operations in theexample process 200 can be performed by a computing device, a server, byanother type of computing system, or any suitable combination of these.In some implementations, the process 200 is executed in a secureenvironment, for example, behind a firewall, in secured hardware, or inanother type of environment. The example process 200 can includeadditional or different operations, and the operations may be executedin the order shown or in a different order. In some implementations, oneor more operations in the example process 200 can be repeated orexecuted in an iterative fashion.

At 210, sample values are obtained from one or more entropy sources. Theentropy sources can include one or more of the entropy sources 104 a-bin FIG. 1, or other types of entropy sources. For example, the samplevalues can be obtained from user inputs, a ring oscillator, a noisydiode, variances in disk read times, and/or system usages.

A probability model may be obtained or derived for each of the entropysources. The probability model for an entropy source can be, or it cancorrespond to, a predetermined set of possible distributions for theentropy source. The distribution for an entropy source indicates, foreach possible output value of the entropy source, a probability that theentropy source will produce that output value. The actual distributionof an entropy source may not be known a priori (e.g., it may change overtime, or it may be impossible or intractable to completely characterizethe actual distribution), and therefore the probability model for anentropy source provides a predetermined set of distributions that couldpotentially describe the entropy source. As such, a predetermined set ofdistributions for an entropy source can be determined from a probabilitymodel for the entropy source or from other types of information.

An example theoretical framework is provided as follows. A probabilityspace Π and a sample space X are sets. In cryptological contexts, X isusually finite but Π is often uncountably infinite. The sample space Xcan be assumed to be finite. An element pϵΠ can be a distribution. Anelement of xϵX can be a sample. A probability function for Π and X canbe a functionP:Π×X→[0,1]:(p,x)→P _(p)(x)such that for all pϵΠ, the following sum holdsΣ_(xϵX) ^(n) P _(p)(x)=1.A probability model can be denoted by a triple (Π, X, P), where Π is aprobability space, X is a sample space, and P is a probability function.In other words, assessing the cryptographic entropy of a source thatproduces sample values in a finite set X, called the sample space, mayin some instances rely on an assumption that the source hasdistributions that belong to some set Π, called the probability space.For an assessment, this assumed probability model can be made explicit.The notation P_(p)(x) can be used to denote P(p,x).

At 220, a typicality for each sample value obtained from the entropysource is determined. The typicality of the obtained sample values canbe determined for each of the distributions in the predetermined set ofdistributions. For example, the typicality of the sample values can bedetermined for some or all of the distributions in a selectedprobability model. In some instances, the typicality for a given samplevalue indicates a probability that a randomly-selected sample value fromthe entropy source system is at most as probable as the given samplevalue. In some implementations at 220, a likelihood for each samplevalue is determined. The likelihood of some or all of the obtainedsample values can be determined for each of the distributions in thepredetermined set of distributions.

At 230, a grading is determined for each of a preselected distributionsbased on the typicalities. In some implementations, the values of thetypicality are used for the grading. In some implementations, the valuesof the likelihood are used for the grading. The grading may bedetermined based on a combination of the typicalities, the likelihoods,or other values derived from the sample values and the distributions. At240, a subset of the distributions is selected, or inferred, based onthe grading. The selection process may include choosing one or moredistributions that have a maximum grading, or that have a grading abovea specified minimum threshold. As such, the selected subset ofdistributions may include a single distribution, or the selected subsetof distributions may include multiple distributions.

Operations 220, 230, and 240 may be described in the example theoreticalframework as follows. A maximally graded technique may be used to selectthe subset of the distributions. Suppose that g is a general grading onprobability model (Π, X, P), associated with a grading-valued inferencefunction i_(g). The maximally graded inference associated with grading gcan be described as a set-valued inference i_(max g) function asfollows:i _(max g)(x)={p:g(x,q)≤g(x,p)∀qϵΠ}The function i_(max g) can be considered as derived from g or fromi_(g). In some cases g is discontinuous and such a maximum p may notexist. In these cases, an alternative may sometimes be available.Consider the supremum of gradings values at x, writtens_(x)=sup_(pϵΠ)g(x,p). Define S_(ϵ)={p: g(x,p)≥s_(x)−ϵ}⊆Π, which arenested according to the size ϵ. As a matter of convenience, S _(ϵ) canbe the closure of S_(ϵ) in some natural topology on Π. If we definei_(supg)(x)=∩_(ϵ>0) S _(ϵ) is non-empty (which is true if Π is given acompact topology), i_(supg)(x) may serve as a suitable substitute for anempty set i_(maxg)(x), even if values of g(x,p)<s_(x) for pϵi_(supg)(x).

In cryptographic applications, entropy parameters may be of moreinterest than the distributions themselves. If the parameters arecontinuous then the definition for i_(supg)(x) above will provide thedesired answer for the parameters. For discontinuous parameters theparameters on the i_(supg)(x) may not be what is desired. In this case,instead of considering i_(supg)(x) as the intersection of the chain ofsets of S _(ϵ)(x) can be considered as the limit of the chain of setsS_(ϵ). This enables us to consider limits of parameters on S_(ϵ), whichmay differ the value of parameters on the intersection. In many cases,the inferred set i_(max g)(x) is a single element (singleton) set. Inthese cases, the inference is much like a point-valued inferencefunction. However, there are often some values of x for which several,possibly infinitely many, different distributions p attain the maximalvalue. If G is a general grading method or I_(G) is grading-valuedinference method, then it is possible to derive a set-valued inferencemethod I_(maxG) using the inference functions above. Maximally gradedinferences can be model-dependent in the sense that the equation fori_(max g) (x) include Π. A potential consequence of thismodel-dependence is that the maximally graded inference in restriction(Θ, X, P) of the model (Π, X, P), may not have a given relation with themaximally graded inference in the model (Π, X, P).

A threshold graded technique or confidence levels may be used to selectthe subset of the distributions. Suppose that g is a general grading ona model (Π, X, P). Let tϵ[0, 1] and call this value the threshold level.The threshold graded inference function i_(g>t) is a set-valuedinference function given byi _(g>t)(x)={p:g(x,p)>t}If t>u, then i_(g>t)(x)⊆i_(g≥u)(x), so the sets obtained are shrinkingor stable in size as a function of the threshold. A high threshold maylead to a narrow, perhaps even empty, inference, while a low thresholdmay lead to a broad inference.

The value c=1−t may sometimes be called the confidence level of theinference. As confidence increases, the breadth of the inference mayincrease (or stay stable). This reflects the notion that as we cangenerally make the sacrifice of broadening our inference set in order togain a more confident inference. In some instances, gradings can be bestthresholded when the distribution of the grading, for fixed p andvarying x, has some resemblance to the uniform distribution on [0, 1],because the confidence level has a better meaning. Some gradings havesuch a property and others do not. Threshold graded inferences are notmodel-dependent provided that the grading is not model-dependent. Inparticular, if i_(Θ)(x) is the threshold graded inference in restriction(Θ, X, P) of the model (Π, X, P), and i_(Π)(x) is the threshold gradedinference in the model (Π, X, P), theni _(Θ)(x)=Θ∩i _(Θ)(x).When using such a threshold graded inference and taking the infima ofparameters, then restriction of the of model cannot decrease theinferred parameter, and relaxing model cannot increase the inferredparameter. In some instances, it may be presumed that an adversary caninfluence the choice of p in Π. If an adversary has such power over p,then a maximally graded inference may have less value. For appropriategradings, a high-confidence threshold grading would still have somevalue.

The grading may be generated by any appropriate technique. Thelikelihood grading g_(L) can be described asg _(L)(x,p)=P _(p)(x).An associated inference function can be written as L_(x)=i_(gL)(x).Therefore L_(x)(p)=P_(p)(x). The term likelihood can indicate that L_(x)may have different properties than a probability function has. Forexample summing (or integrating) the values of L_(x) over allprobability distributions is not guaranteed to yield 1. Likelihood is awell-known and fundamental notion in statistical inference.

The grading may be generated based on typicality. For a giveninclusivity level kϵ[0,1], a typicality grading g_(k) can be written:

${g_{k}( {x,p} )} = {( {\sum\limits_{y:{{P_{p}{(y)}} < {P_{p}{(x)}}}}\;{P_{p}(y)}} ) + {k( {\sum\limits_{{y:{P_{p}{(y)}}} = {P_{p}{(x)}}}\;{P_{p}(y)}} )}}$In some examples, the values

$k \in \{ {0,\frac{1}{2},1} \}$give, balanced and inclusive typicality, respectively. Additional ordifferent values of k may be used. Inclusive typicality g₁(x,p) mayprepresent the probability that a random sample y is at most as probablyas x. Exclusive typicality g₀(x,p) can represent the probability that arandom sample y is less probable than x. Balanced typicality is theaverage of inclusive and exclusive typicality. In other words, balancedtypicality is half-way between inclusive and exclusive.

Typicality, unlike likelihood, when used for inference, attempts tocapture the notion of how a sample compares in probability to othersamples under the same probability distribution. For a fixeddistribution p, ranking sample values x by typicality or likelihoodgives the same ranking. For fixed x, and varying p, the rankings inducedby typicality may differ from those by likelihood. When p is a uniformdistribution on X, then typicality is constant for all x, and takes onthe value k, the inclusivity. When p is an almost uniform distributionon X, then for the most probable x, it takes value approximately1−(1−k)/|X|. For k<1, this will be larger than the typicality at theuniform distribution. When p is subuniform on X, then

${g_{k}( {x,p} )} = \{ \begin{matrix}k & {{{if}\mspace{14mu}{P_{p}(x)}} > 0} \\0 & {{{if}\mspace{14mu}{P_{p}(x)}} = 0}\end{matrix} $As such, in models that admit subuniform distributions, any inferencebased on typicality may treat them equally. Some models may admitdistributions with higher typicality than all subuniform distributions,in which case some useful inferences can be made. In some cases, samplestatistics may serve as tiebreakers between subuniform distributions. Inthe formulation above, inclusive typicality is at least as large aslikelihood:g ₁(x,p)≥g _(L)(x,p)but balanced and inclusive typicality could be less. Similarly,1−g₀(x,p)≥g_(L)(x,p). The gap between exclusive and inclusive typicalitycan be at least the likelihood,g ₁(x,p)−g ₀(x,p)≥g _(L)(x,p)

The notion of typicality is based on well-known notions in statistics ofsignificance level, p-value (also known as percentile or quantile,depending on the units) and cumulative probability function. The generalnotion of significance level refers to a value of the p-value. Thegeneral notion of p-value is a sample statistic that takes uniformdistribution on [0, 1], at least under the null hypothesis. A p-valuestatistic may be formed for continuous distributions by taking acumulative probability function with respect to some function ƒ on thesample space. Any choice function ƒ can yield a p-value. So, the p-valueof x is the probability that ƒ(y)≥ƒ(x), for y drawn from the samedistribution. A common use of p-values is when the distribution is anormal distribution and the function ƒ is the identity, then p-value isrelated to the Gauss error function. In cryptographic applications,distribution are typically discrete, so achieving a proper p-valueuniformly distributed in [0, 1] cannot always be strictly done.Nevertheless such a distribution can be used. In cryptographicapplications, the parameters of interest, entropy and so on, dependprimarily on the distribution, not on specific structure in x.Typicality can be the cumulative probability with respect to theprobability function.

Typicality can rank the sample space by probability. Such ranking isoften implicit in rankings of popularity, such as music sales charts,election results, and lists of popular names. The game show Family Feudinvolves such a ranking. For an almost uniform distribution (with noequal probabilities), the distribution typicality is almost uniformlydistributed in [0, 1], much like a p-value. Randomized typicalityg_(?)(x,p) is a random variable defined with the same equation astypicality above, except that variable k is selected uniformly at randomfrom the interval [0, 1]. If distribution p is fixed, and x has theprobability distribution given by p, then g_(?)(x,p) is uniformlydistributed in [0, 1]. The notion of typicality may adapt the notion ofsignificance level towards the task of assessing min-entropy, in that itranks the samples according to their probabilities.

At 250, entropy is calculated. The calculated entropy can represent, andcan be used as, an assessment of the entropy of the entropy source fromwhich the sample values were obtained. The entropy can be calculatedbased on the selected subset of distributions, the sample values fromthe entropy sources, or any suitable combination of these and otherinformation. The entropy can be calculated based on additional ordifferent factors (e.g., information available to an adversary, etc.).Calculating the entropy can include identifying side channel informationand computational resources available to an adversary. The entropycalculation can include determining, for each distribution in theselected subset of distributions, a logarithm of a maximal probabilityof the adversary guessing the cryptographic secret using both the sidechannel information and the resources available to the adversary. Theminimal value, over all of the distributions, of the logarithm can thenbe identified as the entropy. Additional or different techniques may beused.

Any appropriate entropy measure may be used. For example, themin-entropy, the Shannon entropy, the Renyi entropy, the workingentropy, the contingent applied working entropy, or any suitablecombination or variation thereof may be used to calculate entropy. Theentropy can be expressed in units of bits, or in other units.Min-entropy for a distribution p may be expressed, for example, asH _(∞)(p)=−log₂ max_(x) P _(p)(x)=min_(x)(−log₂ P _(p)(x)).

Renyi entropy at order t for the distribution p may be expressed, forexample, as

${H_{t}(p)} = {\frac{1}{1 - t}\log_{2}{\sum\limits_{x \in X}\;{{P_{p}^{t}(x)}.}}}$As t→1, Renyi entropy approaches Shannon entropy. As t→∞, Renyi entropyapproaches min-entropy.

Working entropy may be expressed, for example, asH _((w))(p)=min_(x) _(j) (−log₂ Σ_(j=1) ^(└2) ^(w) ^(┘) P _(p)(x _(j))),where the minimum is taken over arbitrary sequences x_(j) of distinctvalues. As a convention let P_(p)(x)=0 if x is not an element of X,which allows the sum to be well defined for all w. The variable w is theworkload and can be measured in bits. Min-entropy is the working entropyat workload of zero bits.

In some implementations, the example process 200 may also includedetermining whether the entropy source system has adequate entropy toachieve a specified security level in the cryptography system. Theexample process 200 can modify the entropy source system if it isdetermined that the entropy source system provides inadequate entropy.In some instances, the entropy source system may be modified byincreasing the number of entropy sources. For example, a computingsystem may have access to multiple entropy sources (e.g., mousemovements, variances in disk read time, system process usages, etc.),and one or more of the entropy sources may be accessed or used as neededor as appropriate. In some instances, the entropy source system may bemodified by increasing the size of the output produced by the entropysource system. For example, the acquisition time or the output rate (orboth) of a chronological entropy source may be increased as needed or asappropriate.

The following provides an example implementation of the process 200 inthe context of a particular example probability model that can be usedfor a broad class of entropy sources: chronological sources.Chronological sources can generate elements of a sample vector in achronological fashion for each instantiation. In some cases,chronological sources can be run for varying amounts of time. Theanalyzed example model for these sources is the Markov model for eachinstantiation, or more generally, the common power of the Markov modelwhen considering multiple instantiations of the source. Assessment maybe performed based on additional or different types of models. The totalassessment of entropy can be taken as the minimum assessment over allthe hypothesized models, including the example model provided. In theexample model, it may be advantageous to compute gradings induced from asample statistic. Two example sample statistics are described in moredetail below: the identity sample statistic and the Markov frequencystatistic. In the latter case, the induced probabilities may be computedusing the Goulden-Jackson formula.

An example probability model for a chronological source of entropy isprovided as follows. A chronological source of entropy generates samplesin the form of a sequence (x₀, . . . , x_(m-1)), where each entry in thesequence is a bit string of some length t. The real amount of time thatthe chronological entropy source needs to run in order to produce asequence of length m may vary. In the example probability model, theentropy of a chronological source can be assessed as a function of m.If, in practice, the source can be used by running it for a fixed amountof time, and thereby obtained a variable m, then a model accounting forthis can be formulated. In this fixed time model, the values for m eachhave different probability. Such a model may have benefits. In someexample models if amount of time, or power, needed to produce the samplesequence of length m is likely to be revealed to an adversary in someform of side channel, then this fact may be accounted for by assessingthe contingent entropy.

Therefore, in the example probability model, the probability space isX={0, 1}^(tm). The example model is a Markov model, with each element inthe sample sequence depending only on the previous element. This is theMarkov model of length m and width 2t. Elements of X can be representedin the form (x₀, . . . , x_(m-1)), with each x_(i)ϵ{0, 1}^(t). Theelements of x_(i) are regarded to have been produced by the source inchronological order, but the individual t bits within each x_(i) are notdeemed to have any chronological order. In the example model, the valuesof x_(i) can be treated arbitrarily. The probability space includes apair: a real vector υ of dimension 2^(t) and a real square matrix ofdimensions (2^(t))×(2^(t)). The indices of υ and M may be represented bybit strings of length t, or equivalently as integers ranging from 0 to2^(t)−1 inclusive. The probability function can be given by:

${P_{v,M}( {x_{0},\ldots\mspace{14mu},x_{m - 1}} )} = {v_{x_{0}}{\prod\limits_{i = 1}^{m - 1}\; M_{x_{i - 1},x_{i}}}}$The distributions (υ,M)ϵΠ can meet that conditions that υ_(x)≥0 and

${\sum\limits_{x}\; v_{x}} = 1$ ${\sum\limits_{x}\; M_{y,x}} = 1$with the latter holding over all valid indices y.

The example probability model may, in some instances, further assumethat instantiations of the source are identically and independentlydistributed. These further assumptions may mean that inferences based onobservations of one or more instantations are applicable to otherinstantatiations of the source. This results in a model with a largersample space, as is a common power of the Markov model. Considering kinstantiations results in a model with a sample space X={0,1}^(tmk). Theprobability space is the same. Elements of the larger sample space canbe represented as sequences of sequence of bit strings. However, theouter sequence of length k represents the instantiations, and is notchronological. The inner sequences represent the chronological outputsfrom each instantiation. Each individual output includes t bits, whichthe example model treats arbitrarily. The probability function for thelarger model can be:

${P_{v,M}\lbrack {( {x_{0,0},{x_{0,1}\mspace{11mu}\ldots}\mspace{14mu},x_{0,{m - 1}}} ),\ldots\mspace{14mu},( {x_{{k - 1},0},\ldots\mspace{14mu},x_{{k - 1},{m - 1}}} )} \rbrack} = {\prod\limits_{j = 0}^{k - 1}\;{v_{x_{j,0}}{\prod\limits_{i = 1}^{m - 1}\; M_{x_{j,{i - 1}},x_{j,i}}}}}$

In practice, a key might be generated from a relatively shortsubsequence of the output from a single instantiation of the source.During prospective assessment, observations from multiple instantiationsof the source may be possible, which may permit a stronger inferencethan observation of a single source. Nevertheless, even if only a singleinstantiation is observed during prospective assessment, the assumedlarger probability model means that the inference applies to otherinstantiations of the source.

Formal inference in the example Markov model with parameters such that2^(t)>>(km)² in which the source being assessed may essentially producecompletely uniformly and independently distributed bit strings of lengtht as the elements is very likely to give an inference of zero entropy,whereas the total entropy (over all k instantiations and all bitsproduced) may actually be tmn bits. This discrepancy between theinferred and actual entropy may arise due to the relatively low samplesize. With so few observations of the t-bit elements, they could be allbe distinct and a formal inference method would not exclude thepossibility that Markov model distribution is a deterministicdistribution. To adapt to such a setting, formal inference may be byobserving that the t-bit values obtained from the source may have somestructure. A chronological source has elements representing successivet-bit measurements of some real quantity, such as time, or perhaps theresistance across some noisy diode. In some cases, the numericaldifferences between successive measurements may be small enough suchthat repetitions are likely, even though the differences are still t-bitquantities. So, this means applying a transformation on the samplevalues, which for k=1, looks like:(x ₀ , . . . ,x _(m-1))→(x ₀ ,x ₁ −x ₀ , . . . ,x _(m-1) −x _(m-2))

The differences can be computed mod 2^(t). This transformation isreversible, so it does not lose, or gain, any actual entropy. If most ofthe absolute value of the differences are bounded by 2^(t), and2^(t)<<(km)², then there can be repetitions among the sequences. In thiscase, it may make sense to model the subsequence (x₁−x₀, . . . ,x_(m-1)−x_(m-2)) with the example Markov model. The simpletransformation has the effect of making visible to the formal inferencemethods the underlying mechanism of the source. If the differences varysufficiently to be inconsistent with any deterministic Markovdistribution, then the formal inference methods can infer nonzeroentropy. The effect of the transformation has been to change the modelon the original samples.

Note that this transformation has not modified the actual entropy buthas boosted the assessed entropy. The possibility of such boosts warrantcaution. In particular, one may not want to apply some arbitrarytransformation (to the model) which might artificially, and excessively,boost the entropy assessment. To avoid this, assessment may be based onsimple transformations. In particular, the transformation of the modelmay not be based on the sample. The initial term x₀ contributes toactual entropy, but was excluded from the transformed Markov model,because it was not a difference of elements, so it cannot be expected tohave the same role in the transition matrix. Furthermore, formallyassessing a nonzero entropy for x₀ is only likely for k²>>2^(t). In thiscase, x₀ could be assessed using the independent model.

In prospective assessment, the type of entropy assessed can be acontingent applied working entropy. The contingent applied workingentropy can be based on a probability model (Π, X, P). A leakagefunction g: X→Z can be used. If the source issues sample x, then theadversary will learn g(x). An application function ƒ: X→Y can be used.(The application function can be chosen such that it does not includeany cryptographic functions, but it should include any other functionsapplied to the source samples.) A workload value w, in bits, measuringthe cryptographic capabilities of an adversary can be selected. Thecontingent applied working entropy of distribution pϵΠ can be given by:

${H_{{f{({(w)})}}|g}(p)} = {{- \log_{2}}{\sum\limits_{z \in Z}\;{\max\limits_{{W \in {Y:{W}}} = {\lfloor 2^{w}\rfloor}}{\sum\limits_{y \in W}\;{\sum\limits_{x \in {{f^{- 1}{(y)}}\bigcap{g^{- 1}{(z)}}}}\;{P_{p}(x)}}}}}}$For default choices of the parameters, the leakage function g may be setto a constant function, meaning no leakage, and the workload may be setto 0. Different leakage functions an workload values may be used, asappropriate. The application function can reflect how the sampleactually contributes to the seed. In assessing retrospective entropy, itshould be possible to use eventuated entropy.

A sample statistic is a function x: X→S, from the sample space to astatistic space. A sample statistic induces from a model (Π, X, P)another model (Π, S, Q) such that Q_(p) (y)=Σ_(x:s(x)=y)P_(p)(x). It ispossible to make inferences about Π, as though one was making inferencesin the induced model. That is given observation x, look at s(x) and makeinference for the value of the distribution p characterizing assumingthe induced model. The identity function can be regarded as a samplestatistic.

The two example sample statistics for the smaller, single-instantiationexample model, the Markov model of width 2^(t) and length m are theidentity function and the Markov frequency statistic F, can beF(x)=(e(x), U(x)), where e(x)=e_(x) ₀ is a 2^(t) dimensional vector allof whose entries are zero except for the entry in position x₀ whosevalue is 1 (vector entry indices run from 0 to m−1); and U(x) is an2^(t)×2^(t) matrix with non-negative integer entries U_(y,z) indexed byintegers pair (y,z) such that 0≤y, z≤2^(t) withU _(y,z) =|{i|1≤i≤N−1,x _(i-1) =y,x _(i) =z}|

An example sample statistic F for the larger, multiply instantiatedMarkov model, with k instantiations can be:

${F( {x^{0},\ldots\mspace{14mu},x^{k - 1}} )} = {\sum\limits_{j = 0}^{k - 1}\;{F( x^{j} )}}$where the x^(j) are sequences of length m (with entries of t-bitstrings). Each F term in the sum is a pair of a vector and a matrix, andin the sum, the vectors are added, and the matrices are added. In someimplementations, any sample statistic on the given model may be used,for example, if the sample statistic is model-neutral with respect tothe probability model. In some implementations, sample statistics thatare not model-neutral are not be used. In some implementations, samplestatistics that are selected in a manner dependent on observed samplesare not used.

In some implementations, a grading can include a real-valued function onthe probability space, g: Π

. The set of gradings may be referred to as Γ(Π). A grading-valuedinference function is function i: X→Γ(Π), which may also be thought ofas function i: X×Π→

. When clear from context, a grading-valued inference function can justbe called a grading. The notation i_(x)(p) can be used to indicatei(x,p) (which indicates i(x)(p)).

A set-valued inference function i can be a grading-valued inferencefunction i in which i_(x)(p)ϵ{0, 1} for all (x,p)ϵX×P, and one mayidentify i_(x) with i_(x) ⁻¹(p)={pϵΠ: i_(x)(p)=1}, so that i maps X tosubsets of Π. A set-valued inference i is a graded inference if it isderived from another grading-valued inference j in one of two ways.Graded inference i is maximally graded ifi(x)={p:j _(x)(p)≥j _(x)(q),∀qϵΠ}.It may be noted as i=ĵ. Graded inference i is threshold graded withthreshold t ifi(x)={p:j _(x)(p)≥t}It may be noted as i=j^(≥t). Set-valued inferences can be gradedinferences, and can be derived from real grading valued inferences. Anexample graded inference is the threshold graded. An example thresholdlevel is 2⁻⁴⁰.

Four gradings from which graded inference can be derived are provided asfollows. The likelihood grading is given by L_(x)(p)=P_(p)(x). Thelikelihood grading may not be threshold graded. The inclusive typicalitygrading can be given by

${T_{x}(P)} = {\sum\limits_{y \in {X:{{P_{p}{(y)}} \leq {P_{p}{(x)}}}}}\;{P_{p}(y)}}$The balanced typicality grading can given by

${B_{x}(P)} = {( {\sum\limits_{y \in {X:{{P_{p}{(y)}} < {P_{p}{(x)}}}}}\;{P_{p}(y)}} ) + {\frac{1}{2}( {\sum\limits_{{y \in {X:{P_{p}{(y)}}}} = {P_{p}{(x)}}}\;{P_{p}(y)}} )}}$Balanced typicality is an example real grading. The adjusted likelihoodgrading can be given by

${A_{x}(p)} = {\frac{1}{2}( {1 + {P_{p}(x)} - {\sum\limits_{y}\;{P_{p}(y)}^{2}}} )}$In some cases, the adjusted likelihood grading is not threshold graded.Therefore the allowed set-valued gradings can be {circumflex over(L)},{circumflex over (T)},{circumflex over (B)},Â and T^(≥t) andB^(≥t). An example set-valued grading is B²⁻⁴⁰.

For example, assessment of entropy, given observation x, can be based ona set-valued inference i for the distribution. The assessed entropy canbe calculated asH(x)=inf{H _(ƒ((w))|g)(p):pϵi(x)}.Additional or different entropy models may be used. When assessment ismade using multiple methods, such as by different gradings, differentsample statistics, and even different probability models, the assessedentropy can be taken as the minimum of all the assessments.

In the example model, with the example inference function, there can beassessment for the two default sample statistics. The assessment entropycan at most the minimum of these two assessments. An assessment ofentropy may include this default assessment.

The operations described in this specification can be implemented asoperations performed by a data processing apparatus on data stored onone or more computer-readable storage devices or received from othersources. The term “data processing apparatus” encompasses all kinds ofapparatus, devices, and machines for processing data, including by wayof example a programmable processor, a computer, a system on a chip, ormultiple ones, or combinations, of the foregoing. The apparatus caninclude special purpose logic circuitry, e.g., an FPGA (fieldprogrammable gate array) or an ASIC (application-specific integratedcircuit). The apparatus can also include, in addition to hardware, codethat creates an execution environment for the computer program inquestion, e.g., code that constitutes processor firmware, a protocolstack, a database management system, an operating system, across-platform runtime environment, a virtual machine, or a combinationof one or more of them. The apparatus and execution environment canrealize various different computing model infrastructures, such as webservices, distributed computing and grid computing infrastructures.

A computer program (also known as a program, software, softwareapplication, script, or code) can be written in any form of programminglanguage, including compiled or interpreted languages, declarative orprocedural languages, and it can be deployed in any form, including as astand-alone program or as a module, component, subroutine, object, orother unit suitable for use in a computing environment. A computerprogram may, but need not, correspond to a file in a file system. Aprogram can be stored in a portion of a file that holds other programsor data (e.g., one or more scripts stored in a markup languagedocument), in a single file dedicated to the program in question, or inmultiple coordinated files (e.g., files that store one or more modules,sub-programs, or portions of code). A computer program can be deployedto be executed on one computing device or on multiple computers that arelocated at one site or distributed across multiple sites andinterconnected by a communication network.

The processes and logic flows described in this specification can beperformed by one or more programmable processors executing one or morecomputer programs to perform actions by operating on input data andgenerating output. The processes and logic flows can also be performedby, and apparatus can also be implemented as, special purpose logiccircuitry, e.g., an FPGA (field programmable gate array) or an ASIC(application-specific integrated circuit).

Processors suitable for the execution of a computer program include, byway of example, both general and special purpose microprocessors, andany one or more processors of any kind of digital computing device.Generally, a processor will receive instructions and data from aread-only memory or a random access memory or both. The essentialelements of a computing device are a processor for performing actions inaccordance with instructions and one or more memory devices for storinginstructions and data. Generally, a computing device will also include,or be operatively coupled to receive data from or transfer data to, orboth, one or more storage devices for storing data. However, a computingdevice need not have such devices. Moreover, a computer can be embeddedin another device, e.g., a mobile telephone, a personal digitalassistant (PDA), a mobile audio or video player, a game console, aGlobal Positioning System (GPS) receiver, or a portable storage device(e.g., a universal serial bus (USB) flash drive), to name just a few.Devices suitable for storing computer program instructions and datainclude all forms of non-volatile memory, media and memory devices,including by way of example semiconductor memory devices, e.g., EPROM,EEPROM, and flash memory devices; magnetic disks, e.g., internal harddisks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROMdisks. The processor and the memory can be supplemented by, orincorporated in, special purpose logic circuitry.

To provide for interaction with a user, subject matter described in thisspecification can be implemented on a computer having a display device,e.g., an LCD (liquid crystal display) screen for displaying informationto the user and a keyboard and a pointing device, e.g., touch screen,stylus, mouse, etc. by which the user can provide input to the computer.Other kinds of devices can be used to provide for interaction with auser as well; for example, feedback provided to the user can be any formof sensory feedback, e.g., visual feedback, auditory feedback, ortactile feedback; and input from the user can be received in any form,including acoustic, speech, or tactile input. In addition, a computingdevice can interact with a user by sending documents to and receivingdocuments from a device that is used by the user; for example, bysending web pages to a web browser on a user's client device in responseto requests received from the web browser.

Some of the subject matter described in this specification can beimplemented in a computing system that includes a back-end component,e.g., as a data server, or that includes a middleware component, e.g.,an application server, or that includes a front-end component, e.g., aclient computing device having a graphical user interface or a Webbrowser through which a user can interact with an implementation of thesubject matter described in this specification, or any combination ofone or more such back-end, middleware, or front-end components. Thecomponents of the system can be interconnected by any form or medium ofdigital data communication, e.g., a data network.

The computing system can include clients and servers. A client andserver are generally remote from each other and typically interactthrough a data network. The relationship of client and server arises byvirtue of computer programs running on the respective computers andhaving a client-server relationship to each other. In someimplementations, a server transmits data to a client device. Datagenerated at the client device can be received from the client device atthe server.

While this specification contains many specific implementation details,these should not be construed as limitations on the scope of what may beclaimed, but rather as descriptions of features specific to particularimplementations. Certain features that are described in thisspecification in the context of separate implementations can also beimplemented in combination in a single implementation. Conversely,various features that are described in the context of a singleimplementation can also be implemented in multiple implementationsseparately or in any suitable subcombination. Moreover, althoughfeatures may be described above as acting in certain combinations andeven initially claimed as such, one or more features from a claimedcombination can in some cases be excised from the combination, and theclaimed combination may be directed to a subcombination or variation ofa subcombination.

Similarly, while operations are depicted in the drawings in a particularorder, this should not be understood as requiring that such operationsbe performed in the particular order shown or in sequential order, orthat all illustrated operations be performed, to achieve desirableresults. In certain circumstances, multitasking and parallel processingmay be advantageous. Moreover, the separation of various systemcomponents in the implementations described above should not beunderstood as requiring such separation in all implementations, and itshould be understood that the described program components and systemscan generally be integrated together in a single software product orpackaged into multiple software products.

In a general aspect, entropy is evaluated in a cryptography system.Entropy may be evaluated, for example, in connection with manufacturinga cryptographic apparatus that includes or utilizes an entropy source,in connection with deploying or testing a cryptography system, duringuse of a cryptography system, or in other instances. In some cases, theentropy of an entropy source is evaluated by calculating a quantitativemeasure of entropy. Additionally or alternatively, entropy may beevaluated qualitatively, for example, by determining whether the entropyprovided by an entropy source is sufficient to achieve a specified levelof security in the cryptography system.

In some aspects, sample values are produced by an entropy source system.A typicality is determined for each of the sample values. A grading isdetermined for preselected distributions based on the typicalities ofthe sample values. A subset of the preselected distributions areselected based on the gradings. An entropy of the entropy source systemis calculated based on the subset of the plurality of distributions.

These and other aspects may include one or more of the followingfeatures. Whether the entropy source system has adequate entropy toachieve a specified security level in the cryptography system isdetermined. The entropy source system is modified based on adetermination that the entropy source system has inadequate entropy. Thetypicality for a given sample value indicates a probability that arandomly-selected sample value from the entropy source system is at mostas probable as the given sample value. A pseudorandom number generatoris seeded based on a sample value produced by the entropy source system.A cryptographic secret is obtained based on an output of the seededpseudorandom number generator.

Additionally or alternatively, these and other aspects may include oneor more of the following features. A likelihood for each of the samplevalues is determined. The gradings are determined based on thetypicalities of the sample values and the likelihoods of the samplevalues. Selecting a subset of the distributions based on the gradingsincludes selecting one or more distributions having a maximum grading,selecting one or more distributions having a grading above a specifiedthreshold grading, or any suitable combination of these. Calculating theentropy includes identifying side channel information and computationalresources available to an adversary. Calculating the entropy includesdetermining, for each of the subset of distributions, a logarithm of amaximal probability of the adversary guessing the cryptographic secretusing the side channel information and the resources available to theadversary. Calculating the entropy includes identifying as the entropythe minimal value of the logarithm determined for the subset ofdistributions.

Thus, implementations of the subject matter have been described. Otherimplementations are within the scope of the following claims. In somecases, the actions recited in the claims can be performed in a differentorder and still achieve desirable results. In addition, the processesdepicted in the accompanying figures do not necessarily require theparticular order shown, or sequential order, to achieve desirableresults. In certain implementations, multitasking and parallelprocessing may be advantageous.

The invention claimed is:
 1. A method for assessing entropy in acryptography system, the method comprising: obtaining, by a hardwaredata processing apparatus, a plurality of sample values produced by anentropy source system; determining, by the hardware data processingapparatus, a typicality grading for each of a preselected plurality ofprobability distributions, the typicality gradings providing acomparison among the plurality of probability distributions, thetypicality grading for each individual probability distribution beingdetermined based on the respective probabilities of selecting theplurality of sample values from that individual probabilitydistribution, wherein the typicality grading for each individualprobability distribution is determined based on a cumulative probabilitywith respect to that individual probability distribution, anddetermining the typicality grading for each individual probabilitydistribution comprises calculating${\sum\limits_{y:{{P_{p}{(y)}} \leq {P_{p}{(x)}}}}{P_{p}(y)}},$ whereP_(p)(x) represents a probability of a sample value x of the pluralityof sample values according to the individual probability distribution p,and y:P_(p)(y)≤P_(p)(x) represents values that are at most as probableas the sample value x; selecting, by the hardware data processingapparatus, a subset of the plurality of probability distributions basedon the typicality gradings; calculating, by the hardware data processingapparatus, an entropy of the entropy source system based on the subsetof the plurality of probability distributions, wherein the calculatedentropy is used to determine a security risk of the entropy sourcesystem; determining, by the hardware data processing apparatus, whetherthe entropy source system has adequate entropy to achieve a specifiedsecurity level in the cryptography system; modifying, by the hardwaredata processing apparatus, the entropy source system based on adetermination that the entropy source system has inadequate entropy;encrypting, by the hardware data processing apparatus, a message usingthe modified entropy source system; and transmitting, by the hardwaredata processing apparatus, the encrypted message.
 2. The method of claim1, wherein determining the typicality gradings includes determiningtypicalities for each of the sample values.
 3. The method of claim 2,wherein the typicality for a given sample value indicates a probabilitythat a randomly-selected sample value from the entropy source system isat most as probable as the given sample value.
 4. The method of claim 1,wherein encrypting the message using the modified entropy source systemcomprises: seeding a pseudorandom number generator based on a samplevalue produced by the modified entropy source system; obtaining acryptographic secret based on an output of the seeded pseudorandomnumber generator; and encrypting the message using the cryptographicsecret.
 5. The method of claim 1, comprising determining a typicalityand a likelihood for each of the sample values, wherein the typicalitygradings are determined based on the typicalities of the sample valuesand the likelihoods of the sample values.
 6. The method of claim 1,wherein selecting a subset of the plurality of probability distributionsbased on the typicality gradings includes at least one of: selecting oneor more probability distributions having a maximum typicality grading;or selecting one or more probability distributions having a typicalitygrading above a specified threshold typicality grading.
 7. The method ofclaim 1, wherein calculating the entropy includes: identifying sidechannel information and computational resources available to anadversary; determining, for each probability distribution in the subsetof the plurality of probability distributions, a logarithm of a maximalprobability of the adversary guessing a cryptographic secret using theside channel information and the computational resources available tothe adversary; and identifying as the entropy a minimal value of thelogarithms determined for the subset of the plurality of probabilitydistributions.
 8. A hardware data processing apparatus comprising: anentropy source component; an entropy-assessor component operable to:obtain, by the hardware data processing apparatus, a plurality of samplevalues produced by the entropy source component; determine, by thehardware data processing apparatus, a typicality grading for each of apreselected plurality of probability distributions, the typicalitygradings providing a comparison among the plurality of probabilitydistributions, the typicality grading for each individual probabilitydistribution being determined based on the respective probabilities ofselecting the plurality of sample values from that individualprobability distribution, wherein the typicality grading for eachindividual probability distribution is determined based on a cumulativeprobability with respect to that individual probability distribution,and determining the typicality grading for each individual probabilitydistribution comprises calculating${\sum\limits_{y:{{P_{p}{(y)}} \leq {P_{p}{(x)}}}}{P_{p}(y)}},$ whereP_(p)(x) represents a probability of a sample value x of the pluralityof sample values according to the individual probability distribution p,and y:P_(p)(y)≤P_(p)(x) represents values that are at most as probableas the sample value x; select, by the hardware data processingapparatus, a subset of the plurality of probability distributions basedon the typicality gradings; calculate, by the hardware data processingapparatus, an entropy of the entropy source component based on thesubset of the plurality of probability distributions, wherein thecalculated entropy is used to determine a security risk of the entropysource component; determine, by the hardware data processing apparatus,whether the entropy source component is configured to produce adequateentropy for a specified security level in a cryptography system; andmodify, by the hardware data processing apparatus, the entropy sourcecomponent based on a determination that the entropy source component isnot configured to produce adequate entropy; and a cryptographiccommunication component operable to: encrypt, by the hardware dataprocessing apparatus, a message using the modified entropy sourcesystem; and transmit, by the hardware data processing apparatus, theencrypted message.
 9. The hardware data processing apparatus of claim 8,further comprising: a cryptographic secret generator component seededbased on a value produced by the modified entropy source component; andwherein the cryptographic communication component is further operableto: obtain a cryptographic secret from the cryptographic secretgenerator component; and encrypt the message using the cryptographicsecret.
 10. The hardware data processing apparatus of claim 8, theentropy-assessor component being operable to determine a typicality anda likelihood for each of the sample values, wherein the typicalitygradings are determined based on the typicalities of the sample valuesand the likelihoods of the sample values.
 11. The hardware dataprocessing apparatus of claim 8, wherein calculating the entropyincludes: identifying side channel information and computationalresources available to an adversary; determining, for each of the subsetof the plurality of probability distributions, a logarithm of a maximalprobability of the adversary guessing a cryptographic secret using theside channel information and the computational resources available tothe adversary; and identifying as the entropy a minimal value of thelogarithms determined for the subset of the plurality of probabilitydistributions.
 12. A non-transitory computer-readable medium comprisinginstructions for assessing entropy in a cryptography system, theinstructions operable to perform operations when executed by a hardwaredata processing apparatus, the operations comprising: obtaining, by thehardware data processing apparatus, a plurality of sample valuesproduced by an entropy source system; determining, by the hardware dataprocessing apparatus, a typicality grading for each of a preselectedplurality of probability distributions, the typicality gradingsproviding a comparison among the plurality of probability distributions,the typicality grading for each individual probability distributionbeing determined based on the respective probabilities of selecting theplurality of sample values from that individual probabilitydistribution, wherein the typicality grading for each individualprobability distribution is determined based on a cumulative probabilitywith respect to that individual probability distribution, anddetermining the typicality grading for each individual probabilitydistribution comprises calculating${\sum\limits_{y:{{P_{p}{(y)}} \leq {P_{p}{(x)}}}}{P_{p}(y)}},$ whereP_(p)(x) represents a probability of the sample value x of the pluralityof sample values according to the individual probability distribution p,and y:P_(p)(y)≤P_(p)(x) represents values that are at most as probableas the sample value x; selecting, by the hardware data processingapparatus, a subset of the plurality of probability distributions basedon the typicality gradings; calculating, by the hardware data processingapparatus, an entropy of the entropy source system based on the subsetof the plurality of probability distributions, wherein the calculatedentropy is used to determine a security risk of the entropy sourcesystem; determining, by the hardware data processing apparatus, whetherthe entropy source system has adequate entropy to achieve a specifiedsecurity level in the cryptography system; modifying, by the hardwaredata processing apparatus, the entropy source system based on adetermination that the entropy source system has inadequate entropy;encrypting, by the hardware data processing apparatus, a message usingthe modified entropy source system; and transmitting, by the hardwaredata processing apparatus, the encrypted message.
 13. Thecomputer-readable medium of claim 12, wherein determining a typicalitygrading includes determining typicalities for each of the sample values,and the typicality for a given sample value indicates a probability thata randomly-selected sample value from the entropy source system is atmost as probable as the given sample value.
 14. The computer-readablemedium of claim 12, wherein encrypting the message using the modifiedentropy source system comprises: seeding a pseudorandom number generatorbased on a sample value produced by the modified entropy source system;obtaining a cryptographic secret based on an output of the seededpseudorandom number generator; and encrypting the message using thecryptographic secret.
 15. The computer-readable medium of claim 12, theoperations comprising determining a typicality and a likelihood for eachof the sample values, wherein the typicality gradings are determinedbased on the typicalities of the sample values and the likelihoods ofthe sample values.
 16. The computer-readable medium of claim 12, whereinselecting a subset of the plurality of probability distributions basedon the typicality gradings includes at least one of: selecting one ormore probability distributions having a maximum grading; or selectingone or more probability distributions having a grading above a specifiedthreshold grading.
 17. The computer-readable medium of claim 12, whereincalculating the entropy includes: identifying side channel informationand computational resources available to an adversary; determining, foreach of the subset of the plurality of probability distributions, alogarithm of a maximal probability of the adversary guessing acryptographic secret using the side channel information and thecomputational resources available to the adversary; and identifying asthe entropy a minimal value of the logarithms determined for the subsetof the plurality of probability distributions.